In the framework of traditional knowledge distillation, the teacher network transfers all of its own knowledge to the student network, and there are almost no researches on the transfer of partial knowledge or specific knowledge. Considering that the industrial field has the characteristics of single scene and small number of classifications, the evaluation of recognition performance of neural network models in specific categories need to be focused on. Based on the attention feature transfer distillation algorithm, three specific knowledge learning algorithms were proposed to improve the classification performance of student networks in specific categories. Firstly, the training dataset was filtered for specific classes to exclude other non-specific classes of training data. On this basis, other non-specific classes were treated as background and the background knowledge was suppressed in the distillation process, so as to further reduce the impact of other irrelevant knowledge on specific classes of knowledge. Finally, the network structure was changed, that is the background knowledge was suppressed only at the high-level of the network, and the learning of basic graphic features was retained at the bottom of the network. Experimental results show that the student network trained by a specific knowledge learning algorithm can be as good as or even has better classification performance than a teacher network whose parameter scale is six times of that of the student network in specific category classification.
A new feature selection algorithm using forest optimization algorithm was proposed, which aimed at solving the problems of the traditional feature selection using forest optimization algorithm in the stages of initialization, candidate forest generation and updating. In the algorithm, Pearson correlation coefficient and L1 regularization method were used to replace the random initialization strategy in the initialization stage, the methods of separating good and bad trees and fulfilling the difference were used to solve the problems of incompletion of good and bad trees in the candidate forest generation stage, and trees having the same precision but different dimension with the optimal tree were added to the forest in the updating stage. In the experiments, with the same experimental data and experimental parameters, the proposed algorithm and the traditional feature selection using forest optimization algorithm were used to test the small, medium and large dimension data respectively. The experimental results show that the proposed algorithm is better than the traditional feature selection using forest optimization algorithm in the classification performance and dimension reduction ability on two medium and two large dimension data. The experimental results prove the effectiveness of the proposed algorithm in solving feature selection problems.
Aiming at the difficulty of unsupervised feature learning on defect vibration data of train running part, a method based on Compressed Sensing and Deep Wavelet Neural Network (CS-DWNN) was proposed. Firstly, the collected vibration data of train running part were compressed and sampled by Gauss random matrix. Secondly, a DWNN based on improved Wavelet Auto-Encoder (WAE) was constructed, and the compressed data were directly input into the network for automatic feature extraction layer by layer. Finally, the multi-layer features learned by DWNN were used to train multiple Deep Support Vector Machines (DSVMs) and Deep Forest (DF) classifiers respectively, and the recognition results were integrated. In this method DWNN was employed to automatically mine hidden fault information from compressed data, which was less affected by prior knowledge and subjective influence, and complicated artificial feature extraction process was avoided. The experimental results show that the CS-DWNN method achieves an average diagnostic accuracy of 99.16%, and can effectively identify three common faults in train running part. The fault recognition ability of the proposed method is superior to traditional methods such as Artificial Neural Network (ANN), Support Vector Machine (SVM) and deep learning models such as Deep Belief Network (DBN), Stack De-noised Auto-Encoder (SDAE).
Classical regression algorithms for data set analysis of multiple models have the defects of long calculating time and low detecting accuracy of models. Therefore, a heuristic robust regression analysis method was proposed. This method mimicked the clustering principle of immune system. The B cell network was taken as classifier of data set and memory of model set. Conformity between data and model was used as the classification criteria, which improved the accuracy of the data classification. The extraction process of model set was divided into a parallel iterative trial including clustering, regressing and clustering again, by which the solution of model set was gradually approximated to. The simulation results show that the proposed algorithm needs obviously less calculating time and it has higher detecting accuracy of models than classical ones. According to the results of the eight-model data set analysis in this paper, among the classical algorithms, the best algorithm is the successive extraction algorithm based on Random Sample Consensus (RANSAC). Its mean model detecting accuracy is 90.37% and the calculating time is 53.3947s. The detecting accuracy of those classical algorithms which calculating time is below 0.5s is bellow 1%. By the contrary, the proposed algorithm needs only 0.5094s and its detecting accuracy is 98.25%.